Motivation and goals
In this lab, we will explore some of the data analytical methods needed for the analysis of RNA-Seq data. These cover a wide range of statistical concepts, including
- hypothesis testing and multiple testing
- visualization of large matrices using heatmaps
- clustering and distance metrics
- ordination methods such as PCA
- (gene set) enrichment analysis
Setup
Load Packages
First let’s make sure we have all the needed packages installed.
pkgs_needed = c("dplyr", "ggplot2", "DESeq2", "pasilla", "genefilter",
"pheatmap", "readr", "tibble",
"org.Dm.eg.db", "AnnotationDbi", "gsean", "WGCNA")
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install(setdiff(pkgs_needed, installed.packages()))
Then let’s load the packages.
library("tidyverse")
library("ggplot2")
library("DESeq2")
library("pasilla")
library("genefilter")
library("pheatmap")
Example dataset: pasilla
The pasilla data are from an experiment on Drosophila melanogaster cell
cultures that investigated the effect of RNAi knock-down of the splicing factor
on the cells’ transcriptome. In our case, the data are stored as a rectangular table
in a tab-delimited file that comes within the R package pasilla. We use the function
read.table to read this file and put the data into the R variable counts.
fn = system.file("extdata", "pasilla_gene_counts.tsv",
package = "pasilla", mustWork = TRUE)
countsdf = read.csv(fn, sep = "\t", row.names = "gene_id")
counts = as.matrix(countsdf)
Activity: Use a spreadsheet programme (such as MS Excel) to look at the file.
Question 1: what are the data types of the variables countsdf and counts? What is the difference?
NB: If/when you want to work with other data, you’ll have to prepare (or download) a similar count table and read it into R with similar steps as above. Section 2 of the Bioconductor RNA-Seq workflow gives some useful hints on this; there are also many other online tutorials.
Question 2: what are the dimensions of the counts matrix? Print three random rows from it.
When loading data from a file, a good plausibility check is to print out some of
the data, and maybe not only at the very beginning, but also at some random
point in the middle, as we have done above.
Question 3: what is the interpretation of counts[45, 2]?
There were two experimental conditions, termed untreated and treated in
the header of the count table that we loaded. They correspond to negative
control and to siRNA against the gene pasilla, a nuclear RNA binding protein implicated in splicing. The experimental metadata of the
7 samples in this dataset are provided in a spreadsheet-like
table. Next, we again use the function system.file to locate a file with this information, which is
shipped together with the pasilla package. When you work with your own data,
simply prepare and load the corresponding file, or use some other way to
generate a dataframe like pasillaSampleAnno.
annotationFile = system.file("extdata", "pasilla_sample_annotation.csv",
package = "pasilla", mustWork = TRUE)
pasillaSampleAnno = readr::read_csv(annotationFile)
pasillaSampleAnno
## # A tibble: 7 x 6
## file condition type `number of lane… `total number o…
## <chr> <chr> <chr> <dbl> <chr>
## 1 trea… treated sing… 5 35158667
## 2 trea… treated pair… 2 12242535 (x2)
## 3 trea… treated pair… 2 12443664 (x2)
## 4 untr… untreated sing… 2 17812866
## 5 untr… untreated sing… 6 34284521
## 6 untr… untreated pair… 2 10542625 (x2)
## 7 untr… untreated pair… 2 12214974 (x2)
## # … with 1 more variable: `exon counts` <dbl>
As we see here, the overall dataset was produced in two batches, the first one
consisting of three sequencing libraries that were subjected to single-read
sequencing, the second batch consisting of four libraries for which paired-end
sequencing was used. Let’s convert the relevant columns of pasillaSampleAnno
into factors, overriding the default level ordering (which is alphabetical) by
one that makes more sense to us.
pasillaSampleAnno = mutate(
pasillaSampleAnno,
condition = factor(condition, levels = c("untreated", "treated")),
type = factor(type, levels = c("single-read", "paired-end")))
Question 4: The design is approximately balanced between the factor of interest, condition, and the nuisance factor type. How can you check that? Use the table function.
We use the constructor function DESeqDataSetFromMatrix to create a DESeqDataSet from the matrix counts and the sample annotation dataframe pasillaSampleAnno.
Note how in the code below, we have to put in extra work to match the column names of the counts object with the file column of the pasillaSampleAnno dataframe, in particular, we need to remove the fb that happens to be used in the file column for some reason. Such data wrangling is very common in bioinformatics and data science. One of the reasons for storing the data in a DESeqDataSet object is that we then no longer have to worry about such things.
mt = match(colnames(counts), sub("fb$", "", pasillaSampleAnno$file))
pasilla = DESeqDataSetFromMatrix(
countData = counts,
colData = pasillaSampleAnno[mt, ],
design = ~ condition)
class(pasilla)
## [1] "DESeqDataSet"
## attr(,"package")
## [1] "DESeq2"
is(pasilla, "SummarizedExperiment")
## [1] TRUE
The SummarizedExperiment class –and therefore DESeqDataSet– also
contains facilities for storing annotation of the rows of the count matrix.
For now, we are content with the gene identifiers from the row names of
the counts table.
Question 5: When we constructed our SummarizedExperiment object, we
also saved some column metadata which we had initially stored in
pasillaSampleAnno. With which function can we extract this information again?
(Hint:?SummarizedExperiment)
The DESeq2 method
After these preparations, we are now ready to jump straight into differential
expression analysis. A choice of standard analysis steps are wrapped into a
single function, DESeq.
pasilla = DESeq(pasilla)
## estimating size factors
## estimating dispersions
## gene-wise dispersion estimates
## mean-dispersion relationship
## final dispersion estimates
## fitting model and testing
The DESeq function is simply a wrapper that calls, in order, the functions
estimateSizeFactors, estimateDispersions (dispersion estimation) and
nbinomWaldTest (hypothesis tests for differential abundance). You can
always call these functions individually if you want to modify their behavior
or interject custom steps. Let us look at the results (we use the arrange
function to order the results by p-value, starting with the lowest).
res = results(pasilla)
arrange(as_tibble(res, rownames = "geneid"), pvalue)
## # A tibble: 14,599 x 7
## geneid baseMean log2FoldChange lfcSE stat pvalue padj
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 FBgn00391… 731. -4.62 0.169 -27.4 4.89e-165 4.07e-161
## 2 FBgn00251… 1501. 2.90 0.127 22.8 1.53e-115 6.38e-112
## 3 FBgn00291… 3706. -2.20 0.0970 -22.7 1.33e-113 3.69e-110
## 4 FBgn00033… 4343. -3.18 0.144 -22.2 9.56e-109 1.99e-105
## 5 FBgn00350… 638. -2.56 0.137 -18.6 1.29e- 77 2.14e- 74
## 6 FBgn00398… 262. -4.16 0.233 -17.9 1.26e- 71 1.74e- 68
## 7 FBgn00347… 226. -3.51 0.215 -16.4 3.86e- 60 4.59e- 57
## 8 FBgn00298… 490. -2.45 0.152 -16.1 2.92e- 58 3.03e- 55
## 9 FBgn00000… 342. 2.68 0.183 14.7 9.51e- 49 8.79e- 46
## 10 FBgn00510… 153. 2.33 0.177 13.2 1.31e- 39 1.09e- 36
## # … with 14,589 more rows
Explore the result
Question 6:
Plot the data for the top 2 genes (those with the smallest p-values), as well as for 6 random genes
The histogram of p-values and multiple testing
Question 7:
Plot the histogram of p-values.
The distribution displays two main components: a uniform background with values
between 0 and 1, and a peak of small p-values at the left. The uniform
background corresponds to the non-differentially expressed genes. Usually this
is the majority of genes. The left hand peak corresponds to differentially expressed genes.
The ratio of the level of the background to the height of the peak gives us
a rough indication of the false discovery rate (FDR) that would be associated
with calling the genes in the leftmost bin differentially expressed.
Question 8:
How many p-values are \(\le 0.01\)? Compute the median height of all the bins in the histogram, and divide this by the height of the first (leftmost) bin. What is an interpretation of this quantity? Compare it to the false discovery rate as computed by the Benjamini-Hochberg method.
MA plot
Read the Wikipedia description for MA plots. The plots shows the observed fold change versus the mean of the (size-factor normalized) counts. Logarithmic scaling is used for both axes. Points which fall out of the y-axis range are plotted as triangles. To produce an MA plot for our data, we can use the function plotMA in the DESeq2 package.
plotMA(pasilla, ylim = c( -2, 2))

PCA plot
Question 9:
Use the DESeq2 function plotPCA to produce a two-dimensional ordination of the 7 samples in the dataset. Before doing that, first transform the data with the variance stabilizing transformation provided by DESeq2.
This type of plot is useful for visualizing the overall effect of experimental
covariates and/or to detect batch effects. Here, the first principal axis,
PC1, is mostly aligned with the experimental covariate of interest
(untreated / treated), while the second axis is roughly aligned with
the sequencing protocol (single-read / paired-end). Instead of PCA, other
ordination methods, for instance multi-dimensional scaling, can also be useful.
Heatmaps
Draw a heatmap of the transformed data pas_trsf. Since it’s impractical to show all 14599 rows, only plot the subset of the 30 most variable genes.
Question 10:
If you want, you can try a different heatmap package (for example ComplexHeatmap) and explore a more enriched heatmap plot.
Two-factor analysis
Besides the treatment with siRNA, the pasilla data have another covariate,
type, which indicates the type of sequencing that was performed.
We saw in the PCA plot that this type had a considerable
systematic effect on the data. Our basic analysis did not take this account,
but we will do so now. This should help us get a more correct picture of which
differences in the data are attributable to the treatment, and which are
confounded—or masked—by the sequencing type.
pasillaTwoFactor = pasilla
design(pasillaTwoFactor) = formula(~ type + condition)
pasillaTwoFactor = DESeq(pasillaTwoFactor)
Of the two variables type and condition, the one of primary interest
is the latter, and in DESeq2, the convention is to put it at the end of the
formula. This convention has no effect on the model fitting, but it helps
simplify some of the subsequent results reporting. Again, we access the results
using the results function.
res2 = results(pasillaTwoFactor)
arrange(as_tibble(res2, rownames = "geneid"), pvalue)
## # A tibble: 14,599 x 7
## geneid baseMean log2FoldChange lfcSE stat pvalue padj
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 FBgn00033… 4343. -3.13 0.109 -28.7 1.78e-181 1.57e-177
## 2 FBgn00265… 43909. -2.48 0.0875 -28.3 2.04e-176 8.98e-173
## 3 FBgn00391… 731. -4.62 0.167 -27.7 2.69e-169 7.88e-166
## 4 FBgn00251… 1501. 2.85 0.104 27.4 5.38e-165 1.18e-161
## 5 FBgn00291… 3706. -2.20 0.0961 -22.9 1.27e-115 2.23e-112
## 6 FBgn00350… 638. -2.58 0.129 -20.0 7.55e- 89 1.11e- 85
## 7 FBgn00398… 262. -4.18 0.217 -19.3 5.54e- 83 6.97e- 80
## 8 FBgn00347… 226. -3.54 0.206 -17.2 2.75e- 66 3.03e- 63
## 9 FBgn00298… 490. -2.44 0.148 -16.5 2.13e- 61 2.08e- 58
## 10 FBgn00000… 342. 2.62 0.159 16.5 2.74e- 61 2.41e- 58
## # … with 14,589 more rows
It is also possible to retrieve the \(\log_2\) fold changes, p-values and adjusted
p-values associated with the type variable. The function results takes an
argument contrast that lets users specify the name of the variable, the level
that corresponds to the numerator of the fold change and the level that corresponds
to the denominator of the fold change.
resType = results(pasillaTwoFactor,
contrast = c("type", "single-read", "paired-end"))
arrange(as_tibble(resType, rownames = "geneid"), pvalue)
## # A tibble: 14,599 x 7
## geneid baseMean log2FoldChange lfcSE stat pvalue padj
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 FBgn0003… 6604. -2.55 0.0989 -25.8 3.89e-147 3.33e-143
## 2 FBgn0053… 295. -2.60 0.178 -14.6 2.23e- 48 9.53e- 45
## 3 FBgn0083… 62.7 -4.65 0.426 -10.9 1.11e- 27 2.41e- 24
## 4 FBgn0029… 1611. -3.58 0.329 -10.9 1.13e- 27 2.41e- 24
## 5 FBgn0086… 2753. -1.06 0.0999 -10.6 3.10e- 26 5.30e- 23
## 6 FBgn0028… 1731. 1.19 0.115 10.4 3.97e- 25 5.67e- 22
## 7 FBgn0030… 128. -2.20 0.224 -9.83 8.54e- 23 1.04e- 19
## 8 FBgn0039… 593. 1.10 0.119 9.26 2.00e- 20 2.07e- 17
## 9 FBgn0037… 491. 1.67 0.181 9.25 2.17e- 20 2.07e- 17
## 10 FBgn0053… 139. -11.0 1.20 -9.12 7.52e- 20 6.43e- 17
## # … with 14,589 more rows
So what did we gain from this analysis that took into account type as a
nuisance factor (sometimes also called, more politely, a blocking factor),
compared to the simple comparison between two groups?
Question 11:
Count and compare the number of genes that pass a certain significance threshold in
each of the two analyses.
Question 12:
Make a scatterplot of -log10 of the p-values from both analyses against each other. What do you notice?
A gene set enrichment analysis
We here conduct a basic workflow the purpose of which is to give us some feeling or intuition about the results. This is not hardcore statistics. There are numerous options, subtleties and various software implementations with a wide range of quality.
First, we need to embark in one of the favourite bioinformatics pasttimes, converting gene identifies from one system to another. The function that we use here, gsean from the equinymous package, emits some warnings, which we could dig into, or ignore for now.
library("gsean")
library("org.Dm.eg.db")
statistic = setNames(res2$stat, rownames(res2))
geneid = AnnotationDbi::select(org.Dm.eg.db, names(statistic),
"ENTREZID", "FLYBASE")
exprs_pasilla = counts(pasilla, normalized = TRUE)
stopifnot(identical(geneid$FLYBASE, names(statistic)),
identical(geneid$FLYBASE, rownames(exprs_pasilla)))
names(statistic) = rownames(exprs_pasilla) = geneid$ENTREZID
load(system.file("data", "GO_dme.rda", package = "gsean"))
gsea = gsean(GO_dme, statistic, exprs_pasilla)
p = GSEA.barplot(gsea, category = 'pathway', score = 'NES',
top = 15, pvalue = 'padj', sort = 'padj',
numChar = 110) +
theme(plot.margin = margin(10, 10, 10, 50))
plotly::ggplotly(p)
LS0tCnRpdGxlOiAiRWxlbWVudHMgb2YgUk5BLVNlcSBkYXRhIGFuYWx5c2lzIgpzdWJ0aXRsZTogIkV4ZXJjaXNlIGZvciB0aGUgY291cnNlIERhdGEgQW5hbHlzaXMgZm9yIHRoZSBMaWZlIFNjaWVuY2VzIGF0IHRoZSBVbml2ZXJzaXR5IG9mIEhlaWRlbGJlcmciCmRhdGU6ICIyMDIwLTA2LTA1IChEYXkgMTMpIgphdXRob3I6ICJCcml0dGEgVmVsdGVuIGFuZCBXb2xmZ2FuZyBIdWJlciIKb3V0cHV0OiAKICBCaW9jU3R5bGU6Omh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgY29kZV9kb3dubG9hZDogdHJ1ZQotLS0KCjwhLS0KVGhpcyBsYWIgaXMgbWVhbnQgdG8gcmVuZGVyIGludG8gdHdvIEhUTUwgZmlsZXMuIENsaWNraW5nICpLbml0KiBvbiBpdCBpbiBSU3R1ZGlvIHdpbGwgcHJvZHVjZSBUZXN0aW5nLWFuZC1STkFzZXEuaHRtbCAod2hpY2ggY29udGFpbnMgdGhlIFF1ZXN0aW9ucyBhbmQgQW5zd2VycykgYXMgd2VsbCBhcyBUZXN0aW5nLWFuZC1STkFzZXEtbm9hbnMuUm1kLiBPbiB0aGUgbGF0dGVyLCBwbGVhc2UgcnVuICpLbml0KiBvciBybWFya2Rvd246OnJlbmRlciBhZ2FpbiB0byBvYnRhaW4gVGVzdGluZy1hbmQtUk5BLW5vYW5zLmh0bWwgKHdoaWNoIGNvbnRhaW5zIG9ubHkgdGhlIFF1ZXN0aW9ucyBhbmQgbm8gQW5zd2VycykuCi0tPgoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSwgZmlnLmRpbSA9IGMoKDErc3FydCg1KSkvMiwgMSkgKiA1LCBjYWNoZSA9IFRSVUUsIGF1dG9kZXAgPSBUUlVFKSAKb3B0aW9ucyh3aWR0aCA9IDcwKQpgYGAKCiMgTW90aXZhdGlvbiBhbmQgZ29hbHMKCkluIHRoaXMgbGFiLCB3ZSB3aWxsIGV4cGxvcmUgc29tZSBvZiB0aGUgZGF0YSBhbmFseXRpY2FsIG1ldGhvZHMgbmVlZGVkIGZvciB0aGUgYW5hbHlzaXMgb2YgUk5BLVNlcSBkYXRhLiBUaGVzZSBjb3ZlciBhIHdpZGUgcmFuZ2Ugb2Ygc3RhdGlzdGljYWwgY29uY2VwdHMsIGluY2x1ZGluZwoKLSBoeXBvdGhlc2lzIHRlc3RpbmcgYW5kIG11bHRpcGxlIHRlc3RpbmcKLSB2aXN1YWxpemF0aW9uIG9mIGxhcmdlIG1hdHJpY2VzIHVzaW5nIGhlYXRtYXBzCi0gY2x1c3RlcmluZyBhbmQgZGlzdGFuY2UgbWV0cmljcwotIG9yZGluYXRpb24gbWV0aG9kcyBzdWNoIGFzIFBDQQotIChnZW5lIHNldCkgZW5yaWNobWVudCBhbmFseXNpcwoKCiMgU2V0dXAKCiMjIExvYWQgUGFja2FnZXMKCkZpcnN0IGxldCdzIG1ha2Ugc3VyZSB3ZSBoYXZlIGFsbCB0aGUgbmVlZGVkIHBhY2thZ2VzIGluc3RhbGxlZC4KCmBgYHtyIGluc3RhbGwsIGV2YWwgPSBGQUxTRX0KcGtnc19uZWVkZWQgPSBjKCJkcGx5ciIsICJnZ3Bsb3QyIiwgIkRFU2VxMiIsICJwYXNpbGxhIiwgImdlbmVmaWx0ZXIiLAogICAgICAgICAgICAgICAgInBoZWF0bWFwIiwgInJlYWRyIiwgInRpYmJsZSIsIAogICAgICAgICAgICAgICAgIm9yZy5EbS5lZy5kYiIsICJBbm5vdGF0aW9uRGJpIiwgImdzZWFuIiwgIldHQ05BIikKaWYgKCFyZXF1aXJlTmFtZXNwYWNlKCJCaW9jTWFuYWdlciIsIHF1aWV0bHkgPSBUUlVFKSkKICAgIGluc3RhbGwucGFja2FnZXMoIkJpb2NNYW5hZ2VyIikKQmlvY01hbmFnZXI6Omluc3RhbGwoc2V0ZGlmZihwa2dzX25lZWRlZCwgaW5zdGFsbGVkLnBhY2thZ2VzKCkpKQpgYGAKClRoZW4gbGV0J3MgbG9hZCB0aGUgcGFja2FnZXMuCgpgYGB7ciBsb2FkLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0KbGlicmFyeSgidGlkeXZlcnNlIikKbGlicmFyeSgiZ2dwbG90MiIpCmxpYnJhcnkoIkRFU2VxMiIpCmxpYnJhcnkoInBhc2lsbGEiKQpsaWJyYXJ5KCJnZW5lZmlsdGVyIikKbGlicmFyeSgicGhlYXRtYXAiKQpgYGAKCiMjIEV4YW1wbGUgZGF0YXNldDogcGFzaWxsYQoKVGhlIGBwYXNpbGxhYCBkYXRhIGFyZSBmcm9tIGFuIGV4cGVyaW1lbnQgb24gRHJvc29waGlsYSBtZWxhbm9nYXN0ZXIgY2VsbApjdWx0dXJlcyB0aGF0IGludmVzdGlnYXRlZCB0aGUgZWZmZWN0IG9mIFJOQWkga25vY2stZG93biBvZiB0aGUgc3BsaWNpbmcgZmFjdG9yIApvbiB0aGUgY2VsbHMnIHRyYW5zY3JpcHRvbWUuIEluIG91ciBjYXNlLCB0aGUgZGF0YSBhcmUgc3RvcmVkIGFzIGEgcmVjdGFuZ3VsYXIgdGFibGUgCmluIGEgdGFiLWRlbGltaXRlZCBmaWxlIHRoYXQgY29tZXMgd2l0aGluIHRoZSBSIHBhY2thZ2UgYHBhc2lsbGFgLiBXZSB1c2UgdGhlIGZ1bmN0aW9uIApgcmVhZC50YWJsZWAgdG8gcmVhZCB0aGlzIGZpbGUgYW5kIHB1dCB0aGUgZGF0YSBpbnRvIHRoZSBSIHZhcmlhYmxlIGBjb3VudHNgLgoKYGBge3IgbG9hZHBhcywgcmVzdWx0cyA9ICJoaWRlIiwgZXJyb3IgPSBGQUxTRX0KZm4gPSBzeXN0ZW0uZmlsZSgiZXh0ZGF0YSIsICJwYXNpbGxhX2dlbmVfY291bnRzLnRzdiIsCiAgICAgICAgICAgICAgICAgIHBhY2thZ2UgPSAicGFzaWxsYSIsIG11c3RXb3JrID0gVFJVRSkKY291bnRzZGYgPSByZWFkLmNzdihmbiwgc2VwID0gIlx0Iiwgcm93Lm5hbWVzID0gImdlbmVfaWQiKQpjb3VudHMgPSBhcy5tYXRyaXgoY291bnRzZGYpCmBgYAoKKipBY3Rpdml0eSoqOiBVc2UgYSBzcHJlYWRzaGVldCBwcm9ncmFtbWUgKHN1Y2ggYXMgTVMgRXhjZWwpIHRvIGxvb2sgYXQgdGhlIGZpbGUuCgpgYGB7ciBxdWVzbnVtLCBlY2hvID0gRkFMU0V9CmlxdWVzID0gMApgYGAKCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IHdoYXQgYXJlIHRoZSBkYXRhIHR5cGVzIG9mIHRoZSB2YXJpYWJsZXMgYGNvdW50c2RmYCBhbmQgYGNvdW50c2A/IFdoYXQgaXMgdGhlIGRpZmZlcmVuY2U/CgoKTkI6IElmL3doZW4geW91IHdhbnQgdG8gd29yayB3aXRoIG90aGVyIGRhdGEsIHlvdSdsbCBoYXZlIHRvIHByZXBhcmUgKG9yIGRvd25sb2FkKSBhIHNpbWlsYXIgY291bnQgdGFibGUgYW5kIHJlYWQgaXQgaW50byBSIHdpdGggc2ltaWxhciBzdGVwcyBhcyBhYm92ZS4gU2VjdGlvbiAyIG9mIHRoZSBbQmlvY29uZHVjdG9yIFJOQS1TZXEgd29ya2Zsb3ddKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL3dvcmtmbG93cy92aWduZXR0ZXMvcm5hc2VxR2VuZS9pbnN0L2RvYy9ybmFzZXFHZW5lLmh0bWwpIGdpdmVzIHNvbWUgdXNlZnVsIGhpbnRzIG9uIHRoaXM7IHRoZXJlIGFyZSBhbHNvIG1hbnkgb3RoZXIgb25saW5lIHR1dG9yaWFscy4KCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IHdoYXQgYXJlIHRoZSBkaW1lbnNpb25zIG9mIHRoZSBgY291bnRzYCBtYXRyaXg/IFByaW50IHRocmVlIHJhbmRvbSByb3dzIGZyb20gaXQuCgoKV2hlbiBsb2FkaW5nIGRhdGEgZnJvbSBhIGZpbGUsIGEgZ29vZCBwbGF1c2liaWxpdHkgY2hlY2sgaXMgdG8gcHJpbnQgb3V0IHNvbWUgb2YgCnRoZSBkYXRhLCBhbmQgbWF5YmUgbm90IG9ubHkgYXQgdGhlIHZlcnkgYmVnaW5uaW5nLCBidXQgYWxzbyBhdCBzb21lIHJhbmRvbSAKcG9pbnQgaW4gdGhlIG1pZGRsZSwgYXMgd2UgaGF2ZSBkb25lIGFib3ZlLiAKCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IHdoYXQgaXMgdGhlIGludGVycHJldGF0aW9uIG9mIGBjb3VudHNbNDUsIDJdYD8KCgpUaGVyZSB3ZXJlIHR3byBleHBlcmltZW50YWwgY29uZGl0aW9ucywgdGVybWVkICoqdW50cmVhdGVkKiogYW5kICoqdHJlYXRlZCoqIGluIAp0aGUgaGVhZGVyIG9mIHRoZSBjb3VudCB0YWJsZSB0aGF0IHdlIGxvYWRlZC4gVGhleSBjb3JyZXNwb25kIHRvIG5lZ2F0aXZlCmNvbnRyb2wgYW5kIHRvIHNpUk5BIGFnYWluc3QgdGhlIGdlbmUgcGFzaWxsYSwgYSBudWNsZWFyIFJOQSBiaW5kaW5nIHByb3RlaW4gaW1wbGljYXRlZCBpbiBzcGxpY2luZy4gIFRoZSBleHBlcmltZW50YWwgbWV0YWRhdGEgb2YgdGhlIApgciBuY29sKGNvdW50cylgIHNhbXBsZXMgaW4gdGhpcyBkYXRhc2V0IGFyZSBwcm92aWRlZCBpbiBhIHNwcmVhZHNoZWV0LWxpa2UgCnRhYmxlLiBOZXh0LCB3ZSBhZ2FpbiB1c2UgdGhlIGZ1bmN0aW9uIGBzeXN0ZW0uZmlsZWAgdG8gbG9jYXRlIGEgZmlsZSB3aXRoIHRoaXMgaW5mb3JtYXRpb24sIHdoaWNoIGlzIApzaGlwcGVkIHRvZ2V0aGVyIHdpdGggdGhlIGBwYXNpbGxhYCBwYWNrYWdlLiBXaGVuIHlvdSB3b3JrIHdpdGggeW91ciBvd24gZGF0YSwgCnNpbXBseSBwcmVwYXJlIGFuZCBsb2FkIHRoZSBjb3JyZXNwb25kaW5nIGZpbGUsIG9yIHVzZSBzb21lIG90aGVyIHdheSB0byAKZ2VuZXJhdGUgYSBkYXRhZnJhbWUgbGlrZSBgcGFzaWxsYVNhbXBsZUFubm9gLgoKYGBge3IgYW5ub3RhdGlvbkZpbGUsIG1lc3NhZ2UgPSBGQUxTRX0KYW5ub3RhdGlvbkZpbGUgPSBzeXN0ZW0uZmlsZSgiZXh0ZGF0YSIsICJwYXNpbGxhX3NhbXBsZV9hbm5vdGF0aW9uLmNzdiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGFja2FnZSA9ICJwYXNpbGxhIiwgbXVzdFdvcmsgPSBUUlVFKQpwYXNpbGxhU2FtcGxlQW5ubyA9IHJlYWRyOjpyZWFkX2Nzdihhbm5vdGF0aW9uRmlsZSkKcGFzaWxsYVNhbXBsZUFubm8KYGBgCgpBcyB3ZSBzZWUgaGVyZSwgdGhlIG92ZXJhbGwgZGF0YXNldCB3YXMgcHJvZHVjZWQgaW4gdHdvIGJhdGNoZXMsIHRoZSBmaXJzdCBvbmUgCmNvbnNpc3Rpbmcgb2YgdGhyZWUgc2VxdWVuY2luZyBsaWJyYXJpZXMgdGhhdCB3ZXJlIHN1YmplY3RlZCB0byBzaW5nbGUtcmVhZCAKc2VxdWVuY2luZywgdGhlIHNlY29uZCBiYXRjaCBjb25zaXN0aW5nIG9mIGZvdXIgbGlicmFyaWVzIGZvciB3aGljaCBwYWlyZWQtZW5kIApzZXF1ZW5jaW5nIHdhcyB1c2VkLiAgTGV0J3MgY29udmVydCB0aGUgcmVsZXZhbnQgY29sdW1ucyBvZiBgcGFzaWxsYVNhbXBsZUFubm9gIAppbnRvIGZhY3RvcnMsIG92ZXJyaWRpbmcgdGhlIGRlZmF1bHQgbGV2ZWwgb3JkZXJpbmcgKHdoaWNoIGlzIGFscGhhYmV0aWNhbCkgYnkgCm9uZSB0aGF0IG1ha2VzIG1vcmUgc2Vuc2UgdG8gdXMuCgpgYGB7ciBmYWN0b3JzfQpwYXNpbGxhU2FtcGxlQW5ubyA9IG11dGF0ZSgKICBwYXNpbGxhU2FtcGxlQW5ubywKICBjb25kaXRpb24gPSBmYWN0b3IoY29uZGl0aW9uLCBsZXZlbHMgPSBjKCJ1bnRyZWF0ZWQiLCAidHJlYXRlZCIpKSwKICB0eXBlICAgICAgPSBmYWN0b3IodHlwZSwgbGV2ZWxzID0gYygic2luZ2xlLXJlYWQiLCAicGFpcmVkLWVuZCIpKSkKYGBgCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiBUaGUgZGVzaWduIGlzIGFwcHJveGltYXRlbHkgYmFsYW5jZWQgYmV0d2VlbiB0aGUgZmFjdG9yIG9mIGludGVyZXN0LCBgY29uZGl0aW9uYCwgYW5kIHRoZSBudWlzYW5jZSBmYWN0b3IgYHR5cGVgLiBIb3cgY2FuIHlvdSBjaGVjayB0aGF0PyBVc2UgdGhlIGB0YWJsZWAgZnVuY3Rpb24uCgoKV2UgdXNlIHRoZSBjb25zdHJ1Y3RvciBmdW5jdGlvbiBgREVTZXFEYXRhU2V0RnJvbU1hdHJpeGAgdG8gY3JlYXRlIGEgYERFU2VxRGF0YVNldGAgZnJvbSB0aGUgbWF0cml4IGBjb3VudHNgIGFuZCB0aGUgc2FtcGxlIGFubm90YXRpb24gZGF0YWZyYW1lIGBwYXNpbGxhU2FtcGxlQW5ub2AuCgpOb3RlIGhvdyBpbiB0aGUgY29kZSBiZWxvdywgd2UgaGF2ZSB0byBwdXQgaW4gZXh0cmEgd29yayB0byBtYXRjaCB0aGUgY29sdW1uIG5hbWVzIG9mIHRoZSBgY291bnRzYCBvYmplY3Qgd2l0aCB0aGUgYGZpbGVgIGNvbHVtbiBvZiB0aGUgYHBhc2lsbGFTYW1wbGVBbm5vYCBkYXRhZnJhbWUsIGluIHBhcnRpY3VsYXIsIHdlIG5lZWQgdG8gcmVtb3ZlIHRoZSBgZmJgIHRoYXQgaGFwcGVucyB0byBiZSB1c2VkIGluIHRoZSBgZmlsZWAgY29sdW1uIGZvciBzb21lIHJlYXNvbi4gU3VjaCBkYXRhIHdyYW5nbGluZyBpcyB2ZXJ5IGNvbW1vbiBpbiBiaW9pbmZvcm1hdGljcyBhbmQgZGF0YSBzY2llbmNlLiBPbmUgb2YgdGhlIHJlYXNvbnMgZm9yIHN0b3JpbmcgdGhlIGRhdGEgaW4gYSBgREVTZXFEYXRhU2V0YCBvYmplY3QgaXMgdGhhdCB3ZSB0aGVuIG5vIGxvbmdlciBoYXZlIHRvIHdvcnJ5IGFib3V0IHN1Y2ggdGhpbmdzLgoKYGBge3IgREVTZXEyLCBtZXNzYWdlID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRX0KbXQgPSBtYXRjaChjb2xuYW1lcyhjb3VudHMpLCBzdWIoImZiJCIsICIiLCBwYXNpbGxhU2FtcGxlQW5ubyRmaWxlKSkKcGFzaWxsYSA9IERFU2VxRGF0YVNldEZyb21NYXRyaXgoCiAgY291bnREYXRhID0gY291bnRzLAogIGNvbERhdGEgICA9IHBhc2lsbGFTYW1wbGVBbm5vW210LCBdLAogIGRlc2lnbiAgICA9IH4gY29uZGl0aW9uKQpjbGFzcyhwYXNpbGxhKQppcyhwYXNpbGxhLCAiU3VtbWFyaXplZEV4cGVyaW1lbnQiKQpgYGAKClRoZSBgU3VtbWFyaXplZEV4cGVyaW1lbnRgIGNsYXNzIC0tYW5kIHRoZXJlZm9yZSBgREVTZXFEYXRhU2V0YC0tIGFsc28KY29udGFpbnMgZmFjaWxpdGllcyBmb3Igc3RvcmluZyBhbm5vdGF0aW9uIG9mIHRoZSByb3dzIG9mIHRoZSBjb3VudCBtYXRyaXguIApGb3Igbm93LCB3ZSBhcmUgY29udGVudCB3aXRoIHRoZSBnZW5lIGlkZW50aWZpZXJzIGZyb20gdGhlIHJvdyBuYW1lcyBvZiAKdGhlIGBjb3VudHNgIHRhYmxlLgoKKipRdWVzdGlvbiBgciAoaXF1ZXMgPSBpcXVlcysxKWAqKjogV2hlbiB3ZSBjb25zdHJ1Y3RlZCBvdXIgYFN1bW1hcml6ZWRFeHBlcmltZW50YCBvYmplY3QsIHdlIAphbHNvIHNhdmVkIHNvbWUgY29sdW1uIG1ldGFkYXRhIHdoaWNoIHdlIGhhZCBpbml0aWFsbHkgc3RvcmVkIGluIApgcGFzaWxsYVNhbXBsZUFubm9gLiBXaXRoIHdoaWNoIGZ1bmN0aW9uIGNhbiB3ZSBleHRyYWN0IHRoaXMgaW5mb3JtYXRpb24gYWdhaW4/CihIaW50OmA/U3VtbWFyaXplZEV4cGVyaW1lbnRgKQoKCiMgVGhlIERFU2VxMiBtZXRob2QKCkFmdGVyIHRoZXNlIHByZXBhcmF0aW9ucywgd2UgYXJlIG5vdyByZWFkeSB0byBqdW1wIHN0cmFpZ2h0IGludG8gZGlmZmVyZW50aWFsIApleHByZXNzaW9uIGFuYWx5c2lzLiBBIGNob2ljZSBvZiBzdGFuZGFyZCBhbmFseXNpcyBzdGVwcyBhcmUgd3JhcHBlZCBpbnRvIGEKc2luZ2xlIGZ1bmN0aW9uLCBgREVTZXFgLgoKYGBge3IgZGVzZXEsIG1lc3NhZ2UgPSBUUlVFfQpwYXNpbGxhID0gREVTZXEocGFzaWxsYSkKYGBgCgpUaGUgYERFU2VxYCBmdW5jdGlvbiBpcyBzaW1wbHkgYSB3cmFwcGVyIHRoYXQgY2FsbHMsIGluIG9yZGVyLCB0aGUgZnVuY3Rpb25zIApgZXN0aW1hdGVTaXplRmFjdG9yc2AsIGBlc3RpbWF0ZURpc3BlcnNpb25zYCAoZGlzcGVyc2lvbiBlc3RpbWF0aW9uKSBhbmQgCmBuYmlub21XYWxkVGVzdGAgKGh5cG90aGVzaXMgdGVzdHMgZm9yIGRpZmZlcmVudGlhbCBhYnVuZGFuY2UpLiBZb3UgY2FuCmFsd2F5cyBjYWxsIHRoZXNlIGZ1bmN0aW9ucyBpbmRpdmlkdWFsbHkgaWYgeW91IHdhbnQgdG8gbW9kaWZ5IHRoZWlyIGJlaGF2aW9yCm9yIGludGVyamVjdCBjdXN0b20gc3RlcHMuIExldCB1cyBsb29rIGF0IHRoZSByZXN1bHRzICh3ZSB1c2UgdGhlIGBhcnJhbmdlYCAKZnVuY3Rpb24gdG8gb3JkZXIgdGhlIHJlc3VsdHMgYnkgcC12YWx1ZSwgc3RhcnRpbmcgd2l0aCB0aGUgbG93ZXN0KS4KCmBgYHtyIHRoZXJlc3VsdHN9CnJlcyA9IHJlc3VsdHMocGFzaWxsYSkKYXJyYW5nZShhc190aWJibGUocmVzLCByb3duYW1lcyA9ICJnZW5laWQiKSwgcHZhbHVlKQpgYGAKCiMgRXhwbG9yZSB0aGUgcmVzdWx0CgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKUGxvdCB0aGUgZGF0YSBmb3IgdGhlIHRvcCAyIGdlbmVzICh0aG9zZSB3aXRoIHRoZSBzbWFsbGVzdCBwLXZhbHVlcyksIGFzIHdlbGwgYXMgZm9yIDYgcmFuZG9tIGdlbmVzCgoKIyBUaGUgaGlzdG9ncmFtIG9mIHAtdmFsdWVzIGFuZCBtdWx0aXBsZSB0ZXN0aW5nCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKUGxvdCB0aGUgaGlzdG9ncmFtIG9mIHAtdmFsdWVzLgoKICAKVGhlIGRpc3RyaWJ1dGlvbiBkaXNwbGF5cyB0d28gbWFpbiBjb21wb25lbnRzOiBhIHVuaWZvcm0gYmFja2dyb3VuZCB3aXRoIHZhbHVlcyAKYmV0d2VlbiAwIGFuZCAxLCBhbmQgYSBwZWFrIG9mIHNtYWxsIHAtdmFsdWVzIGF0IHRoZSBsZWZ0LiAgVGhlIHVuaWZvcm0gCmJhY2tncm91bmQgY29ycmVzcG9uZHMgdG8gdGhlIG5vbi1kaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZ2VuZXMuIFVzdWFsbHkgdGhpcyAKaXMgdGhlIG1ham9yaXR5IG9mIGdlbmVzLiBUaGUgbGVmdCBoYW5kIHBlYWsgY29ycmVzcG9uZHMgdG8gZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIGdlbmVzLgoKVGhlIHJhdGlvIG9mIHRoZSBsZXZlbCBvZiB0aGUgYmFja2dyb3VuZCB0byB0aGUgaGVpZ2h0IG9mIHRoZSBwZWFrIGdpdmVzIHVzIAphIHJvdWdoIGluZGljYXRpb24gb2YgdGhlIGZhbHNlIGRpc2NvdmVyeSByYXRlIChGRFIpIHRoYXQgd291bGQgYmUgYXNzb2NpYXRlZCAKd2l0aCBjYWxsaW5nIHRoZSBnZW5lcyBpbiB0aGUgbGVmdG1vc3QgYmluIGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZC4KCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IAoKSG93IG1hbnkgcC12YWx1ZXMgYXJlICRcbGUgMC4wMSQ/IENvbXB1dGUgdGhlIG1lZGlhbiBoZWlnaHQgb2YgYWxsIHRoZSBiaW5zIGluIHRoZSBoaXN0b2dyYW0sIGFuZCBkaXZpZGUgdGhpcyBieSB0aGUgaGVpZ2h0IG9mIHRoZSBmaXJzdCAobGVmdG1vc3QpIGJpbi4gV2hhdCBpcyBhbiBpbnRlcnByZXRhdGlvbiBvZiB0aGlzIHF1YW50aXR5PyBDb21wYXJlIGl0IHRvIHRoZSBmYWxzZSBkaXNjb3ZlcnkgcmF0ZSBhcyBjb21wdXRlZCBieSB0aGUgQmVuamFtaW5pLUhvY2hiZXJnIG1ldGhvZC4KCiAgCgojIE1BIHBsb3QKClJlYWQgdGhlIFdpa2lwZWRpYSBkZXNjcmlwdGlvbiBmb3IgW01BIHBsb3RzXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9NQV9wbG90KS4gVGhlIHBsb3RzIHNob3dzIHRoZSBvYnNlcnZlZCBmb2xkIGNoYW5nZSB2ZXJzdXMgdGhlIG1lYW4gb2YgdGhlIChzaXplLWZhY3RvciBub3JtYWxpemVkKSBjb3VudHMuIExvZ2FyaXRobWljIHNjYWxpbmcgaXMgdXNlZCBmb3IgYm90aCBheGVzLiBQb2ludHMgd2hpY2ggZmFsbCBvdXQgb2YgdGhlIHktYXhpcyByYW5nZSBhcmUgcGxvdHRlZCBhcyB0cmlhbmdsZXMuIFRvIHByb2R1Y2UgYW4gTUEgcGxvdCBmb3Igb3VyIGRhdGEsIHdlIGNhbiB1c2UgdGhlIGZ1bmN0aW9uIGBwbG90TUFgIGluIHRoZSBgREVTZXEyYCBwYWNrYWdlLgoKYGBge3IgTUF9CnBsb3RNQShwYXNpbGxhLCB5bGltID0gYyggLTIsIDIpKQpgYGAKCiMgUENBIHBsb3QKCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IApVc2UgdGhlIGBERVNlcTJgIGZ1bmN0aW9uIGBwbG90UENBYCB0byBwcm9kdWNlIGEgdHdvLWRpbWVuc2lvbmFsIG9yZGluYXRpb24gb2YgdGhlIGByIG5jb2wocGFzaWxsYSlgIHNhbXBsZXMgaW4gdGhlIGRhdGFzZXQuIEJlZm9yZSBkb2luZyB0aGF0LCBmaXJzdCB0cmFuc2Zvcm0gdGhlIGRhdGEgd2l0aCB0aGUgdmFyaWFuY2Ugc3RhYmlsaXppbmcgdHJhbnNmb3JtYXRpb24gcHJvdmlkZWQgYnkgYERFU2VxMmAuIAoKClRoaXMgdHlwZSBvZiBwbG90IGlzIHVzZWZ1bCBmb3IgdmlzdWFsaXppbmcgdGhlIG92ZXJhbGwgZWZmZWN0IG9mIGV4cGVyaW1lbnRhbCAKY292YXJpYXRlcyBhbmQvb3IgdG8gZGV0ZWN0IGJhdGNoIGVmZmVjdHMuIEhlcmUsIHRoZSBmaXJzdCBwcmluY2lwYWwgYXhpcywKUEMxLCBpcyBtb3N0bHkgYWxpZ25lZCB3aXRoIHRoZSBleHBlcmltZW50YWwgY292YXJpYXRlIG9mIGludGVyZXN0IAoodW50cmVhdGVkIC8gdHJlYXRlZCksIHdoaWxlIHRoZSBzZWNvbmQgYXhpcyBpcyByb3VnaGx5IGFsaWduZWQgd2l0aCAKdGhlIHNlcXVlbmNpbmcgcHJvdG9jb2wgKHNpbmdsZS1yZWFkIC8gcGFpcmVkLWVuZCkuIEluc3RlYWQgb2YgUENBLCBvdGhlciAKb3JkaW5hdGlvbiBtZXRob2RzLCBmb3IgaW5zdGFuY2UgbXVsdGktZGltZW5zaW9uYWwgc2NhbGluZywgY2FuIGFsc28gYmUgdXNlZnVsLgoKIyBIZWF0bWFwcwoKRHJhdyBhIGhlYXRtYXAgb2YgdGhlIHRyYW5zZm9ybWVkIGRhdGEgYHBhc190cnNmYC4gU2luY2UgaXQncyBpbXByYWN0aWNhbCB0byBzaG93IGFsbCBgciBucm93KHBhc2lsbGEpYCByb3dzLCBvbmx5IHBsb3QgdGhlIHN1YnNldCBvZiB0aGUgMzAgbW9zdCB2YXJpYWJsZSBnZW5lcy4KCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IAogIAogIApJZiB5b3Ugd2FudCwgeW91IGNhbiB0cnkgYSBkaWZmZXJlbnQgaGVhdG1hcCBwYWNrYWdlIChmb3IgZXhhbXBsZSBgQ29tcGxleEhlYXRtYXBgKSBhbmQgZXhwbG9yZSBhIG1vcmUgZW5yaWNoZWQgaGVhdG1hcCBwbG90LgoKIyBUd28tZmFjdG9yIGFuYWx5c2lzCgpCZXNpZGVzIHRoZSB0cmVhdG1lbnQgd2l0aCBzaVJOQSwgdGhlIGBwYXNpbGxhYCBkYXRhIGhhdmUgYW5vdGhlciBjb3ZhcmlhdGUsCmB0eXBlYCwgd2hpY2ggaW5kaWNhdGVzIHRoZSB0eXBlIG9mIHNlcXVlbmNpbmcgdGhhdCB3YXMgcGVyZm9ybWVkLgpXZSBzYXcgaW4gdGhlIFBDQSBwbG90IHRoYXQgdGhpcyBgdHlwZWAgaGFkIGEgY29uc2lkZXJhYmxlIApzeXN0ZW1hdGljIGVmZmVjdCBvbiB0aGUgZGF0YS4gT3VyIGJhc2ljIGFuYWx5c2lzIGRpZCBub3QgdGFrZSB0aGlzIGFjY291bnQsIApidXQgd2Ugd2lsbCBkbyBzbyBub3cuIFRoaXMgc2hvdWxkIGhlbHAgdXMgZ2V0IGEgbW9yZSBjb3JyZWN0IHBpY3R1cmUgb2Ygd2hpY2gKZGlmZmVyZW5jZXMgaW4gdGhlIGRhdGEgYXJlIGF0dHJpYnV0YWJsZSB0byB0aGUgdHJlYXRtZW50LCBhbmQgd2hpY2ggYXJlCmNvbmZvdW5kZWQtLS1vciBtYXNrZWQtLS1ieSB0aGUgc2VxdWVuY2luZyB0eXBlLgoKYGBge3IgcmVwbGFjZURlc2lnbiwgbWVzc2FnZSA9IEZBTFNFLCByZXN1bHRzID0gImhpZGUifQpwYXNpbGxhVHdvRmFjdG9yID0gcGFzaWxsYQpkZXNpZ24ocGFzaWxsYVR3b0ZhY3RvcikgPSBmb3JtdWxhKH4gdHlwZSArIGNvbmRpdGlvbikKcGFzaWxsYVR3b0ZhY3RvciA9IERFU2VxKHBhc2lsbGFUd29GYWN0b3IpCmBgYAoKT2YgdGhlIHR3byB2YXJpYWJsZXMgYHR5cGVgIGFuZCBgY29uZGl0aW9uYCwgdGhlIG9uZSBvZiBwcmltYXJ5IGludGVyZXN0CmlzIHRoZSBsYXR0ZXIsIGFuZCBpbiBgREVTZXEyYCwgdGhlIGNvbnZlbnRpb24gaXMgdG8gcHV0IGl0IGF0IHRoZSBlbmQgb2YgdGhlCmZvcm11bGEuIFRoaXMgY29udmVudGlvbiBoYXMgbm8gZWZmZWN0IG9uIHRoZSBtb2RlbCBmaXR0aW5nLCBidXQgaXQgaGVscHMgCnNpbXBsaWZ5IHNvbWUgb2YgdGhlIHN1YnNlcXVlbnQgcmVzdWx0cyByZXBvcnRpbmcuIEFnYWluLCB3ZSBhY2Nlc3MgdGhlIHJlc3VsdHMgCnVzaW5nIHRoZSBgcmVzdWx0c2AgZnVuY3Rpb24uCgpgYGB7ciBtdWx0aVJlc3VsdHN9CnJlczIgPSByZXN1bHRzKHBhc2lsbGFUd29GYWN0b3IpCmFycmFuZ2UoYXNfdGliYmxlKHJlczIsIHJvd25hbWVzID0gImdlbmVpZCIpLCBwdmFsdWUpCmBgYAoKSXQgaXMgYWxzbyBwb3NzaWJsZSB0byByZXRyaWV2ZSB0aGUgJFxsb2dfMiQgZm9sZCBjaGFuZ2VzLCBwLXZhbHVlcyBhbmQgYWRqdXN0ZWQKcC12YWx1ZXMgYXNzb2NpYXRlZCB3aXRoIHRoZSBgdHlwZWAgdmFyaWFibGUuICBUaGUgZnVuY3Rpb24gYHJlc3VsdHNgIHRha2VzIGFuCmFyZ3VtZW50IGBjb250cmFzdGAgdGhhdCBsZXRzIHVzZXJzIHNwZWNpZnkgdGhlIG5hbWUgb2YgdGhlIHZhcmlhYmxlLCB0aGUgbGV2ZWwKdGhhdCBjb3JyZXNwb25kcyB0byB0aGUgbnVtZXJhdG9yIG9mIHRoZSBmb2xkIGNoYW5nZSBhbmQgdGhlIGxldmVsIHRoYXQgY29ycmVzcG9uZHMKdG8gdGhlIGRlbm9taW5hdG9yIG9mIHRoZSBmb2xkIGNoYW5nZS4KCmBgYHtyIG11bHRpVHlwZVJlc3VsdHN9CnJlc1R5cGUgPSByZXN1bHRzKHBhc2lsbGFUd29GYWN0b3IsIAogICAgICAgICAgICAgICAgICBjb250cmFzdCA9IGMoInR5cGUiLCAic2luZ2xlLXJlYWQiLCAicGFpcmVkLWVuZCIpKQoKYXJyYW5nZShhc190aWJibGUocmVzVHlwZSwgcm93bmFtZXMgPSAiZ2VuZWlkIiksIHB2YWx1ZSkKYGBgCgpTbyB3aGF0IGRpZCB3ZSBnYWluIGZyb20gdGhpcyBhbmFseXNpcyB0aGF0IHRvb2sgaW50byBhY2NvdW50IGB0eXBlYCBhcyBhIApudWlzYW5jZSBmYWN0b3IgKHNvbWV0aW1lcyBhbHNvIGNhbGxlZCwgbW9yZSBwb2xpdGVseSwgYSBibG9ja2luZyBmYWN0b3IpLCAKY29tcGFyZWQgdG8gdGhlIHNpbXBsZSBjb21wYXJpc29uIGJldHdlZW4gdHdvIGdyb3Vwcz8gCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKQ291bnQgYW5kIGNvbXBhcmUgdGhlIG51bWJlciBvZiBnZW5lcyB0aGF0IHBhc3MgYSBjZXJ0YWluIHNpZ25pZmljYW5jZSB0aHJlc2hvbGQgaW4gCmVhY2ggb2YgdGhlIHR3byBhbmFseXNlcy4KCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKTWFrZSBhIHNjYXR0ZXJwbG90IG9mIC1sb2cxMCBvZiB0aGUgcC12YWx1ZXMgZnJvbSBib3RoIGFuYWx5c2VzIGFnYWluc3QgZWFjaCBvdGhlci4gV2hhdCBkbyB5b3Ugbm90aWNlPwogIAoKIyBBIGdlbmUgc2V0IGVucmljaG1lbnQgYW5hbHlzaXMKCldlIGhlcmUgY29uZHVjdCBhIGJhc2ljIHdvcmtmbG93IHRoZSBwdXJwb3NlIG9mIHdoaWNoIGlzIHRvIGdpdmUgdXMgc29tZSBmZWVsaW5nIG9yIGludHVpdGlvbiBhYm91dCB0aGUgcmVzdWx0cy4gVGhpcyBpcyBub3QgaGFyZGNvcmUgc3RhdGlzdGljcy4gVGhlcmUgYXJlIG51bWVyb3VzIG9wdGlvbnMsIHN1YnRsZXRpZXMgYW5kIHZhcmlvdXMgc29mdHdhcmUgaW1wbGVtZW50YXRpb25zIHdpdGggYSB3aWRlIHJhbmdlIG9mIHF1YWxpdHkuIAoKRmlyc3QsIHdlIG5lZWQgdG8gZW1iYXJrIGluIG9uZSBvZiB0aGUgZmF2b3VyaXRlIGJpb2luZm9ybWF0aWNzIHBhc3R0aW1lcywgY29udmVydGluZyBnZW5lIGlkZW50aWZpZXMgZnJvbSBvbmUgc3lzdGVtIHRvIGFub3RoZXIuIFRoZSBmdW5jdGlvbiB0aGF0IHdlIHVzZSBoZXJlLCBgZ3NlYW5gIGZyb20gdGhlIGVxdWlueW1vdXMgcGFja2FnZSwgZW1pdHMgc29tZSB3YXJuaW5ncywgd2hpY2ggd2UgY291bGQgZGlnIGludG8sIG9yIGlnbm9yZSBmb3Igbm93LgoKYGBge3Igb3JnZG0sIG1lc3NhZ2UgPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFLCByZXN1bHRzID0gImhpZGUifQpsaWJyYXJ5KCJnc2VhbiIpCmxpYnJhcnkoIm9yZy5EbS5lZy5kYiIpCgpzdGF0aXN0aWMgPSBzZXROYW1lcyhyZXMyJHN0YXQsIHJvd25hbWVzKHJlczIpKQpnZW5laWQgPSBBbm5vdGF0aW9uRGJpOjpzZWxlY3Qob3JnLkRtLmVnLmRiLCBuYW1lcyhzdGF0aXN0aWMpLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJFTlRSRVpJRCIsICJGTFlCQVNFIikKZXhwcnNfcGFzaWxsYSA9IGNvdW50cyhwYXNpbGxhLCBub3JtYWxpemVkID0gVFJVRSkKc3RvcGlmbm90KGlkZW50aWNhbChnZW5laWQkRkxZQkFTRSwgbmFtZXMoc3RhdGlzdGljKSksIAogICAgICAgICAgaWRlbnRpY2FsKGdlbmVpZCRGTFlCQVNFLCByb3duYW1lcyhleHByc19wYXNpbGxhKSkpCgpuYW1lcyhzdGF0aXN0aWMpID0gcm93bmFtZXMoZXhwcnNfcGFzaWxsYSkgPSBnZW5laWQkRU5UUkVaSUQKCmxvYWQoc3lzdGVtLmZpbGUoImRhdGEiLCAiR09fZG1lLnJkYSIsIHBhY2thZ2UgPSAiZ3NlYW4iKSkKZ3NlYSA9IGdzZWFuKEdPX2RtZSwgc3RhdGlzdGljLCBleHByc19wYXNpbGxhKQpwID0gR1NFQS5iYXJwbG90KGdzZWEsIGNhdGVnb3J5ID0gJ3BhdGh3YXknLCBzY29yZSA9ICdORVMnLCAKICAgICAgICAgICAgIHRvcCA9IDE1LCBwdmFsdWUgPSAncGFkaicsIHNvcnQgPSAncGFkaicsIAogICAgICAgICAgICAgbnVtQ2hhciA9IDExMCkgKyAKICB0aGVtZShwbG90Lm1hcmdpbiA9IG1hcmdpbigxMCwgMTAsIDEwLCA1MCkpCmBgYApgYGB7ciBnc2VhcGxvdCwgZmlnLmRpbSA9IGMoMTAsIDMuNSl9CnBsb3RseTo6Z2dwbG90bHkocCkKYGBgCgoKYGBge3Igd2l0aG91dHNvbHV0aW9ucywgZWNobyA9IEZBTFNFfQpyZW1vdmVBbnN3ZXJzID0gZnVuY3Rpb24oeCwgYmVnaW4gPSAiXjxkaXYgY2xhc3M9XCJhbnN3ZXJcIj4kIiwgZW5kID0gIl48L2Rpdj4kIikgewogIGkxID0gZ3JlcChiZWdpbiwgeCkKICBpMiA9IGdyZXAoZW5kLCB4KQogIHN0b3BpZm5vdChsZW5ndGgoaTEpPT1sZW5ndGgoaTIpLCBhbGwoaTE8aTIpKQogIHIgPSB1bmxpc3QobWFwcGx5KGA6YCwgaTEsIGkyLCBTSU1QTElGWSAgPSBGQUxTRSkpIAogIGlmIChsZW5ndGgocikgPiAwKSB4Wy1yXSBlbHNlIHgKfQpmaWxlcyA9IGMoIlRlc3RpbmctYW5kLVJOQXNlcS5SbWQiLCAiVGVzdGluZy1hbmQtUk5BLW5vYW5zLlJtZCIpCnJlYWRMaW5lcyhmaWxlc1sxXSkgJT4lIAogIHJlbW92ZUFuc3dlcnMgJT4lCiAgd3JpdGVMaW5lcyhmaWxlc1syXSkKIyBybWFya2Rvd246OnJlbmRlcihmaWxlc1syXSkKYGBgCgojIyBMaXRlcmF0dXJlCgpbTW9kZXJuIFN0YXRpc3RpY3MgZm9yIE1vZGVybiBCaW9sb2d5IGJ5IFN1c2FuIEhvbG1lcyBhbmQgV29sZmdhbmcgSHViZXIuIENoYXB0ZXIgODogSGlnaC1UaHJvdWdocHV0IENvdW50IERhdGFdKGh0dHBzOi8vd3d3Lmh1YmVyLmVtYmwuZGUvbXNtYi9DaGFwLUNvdW50RGF0YS5odG1sKS4K